1. Field of the Dislosure
The present invention relates to a distributed file system and an execution method thereof, and more particularly, to a system and a method for effectively creating a file by partitioning a large-scale file volume into chunks each of which has a predetermined size and storing the partitioned chunks in a distributed file system.
2. Discussion of Related Art
In recent years, a distributed file system has been applied to the field of computing in which a small number of metadata servers are connected to a plurality of data servers via a network.
The metadata server stores metadata, such as the name of a file, a time of creating the file, and information about a location where actual data of the file is stored, and the data server stores actual data of the file in a distributed manner.
A large-scale file volume is partitioned into chunks of a predetermined size and stored in a distributed file system in a distributed manner, so that the large-scale file volume is effectively stored in a plurality of servers and the availability of file processing is enhanced.
In order to store file data in a distributed file system, an empty chunk is allocated to a data server which is selected by a metadata server, and the file data is written in the chunk.
When a device or network failure occurs during allocation of chunks as part of a process of storing files in a distributed file system, the allocated chunk is not returned and related tasks may be slowed down and a communication socket may be blocked.
In detail, during execution of an input/output operation with respect to a file in a distributed file system, a metadata server may request a data server to perform a certain operation if necessary, and the data server may execute the operation requested by the metadata server and return a result of a calculation to the metadata server. For example, when the metadata server requests the data server to allocate a new chunk and the data server fails to return a result value, the speed of storing a file is reduced and data integrity may not be maintained.
In addition, when a metadata server processes metadata-related operations in transactions while executing a transaction, the metadata server may request a data server to perform an operation, such as allocating of an empty chunk, and a result of the operation may not be returned, which may not allow the transaction of the metadata server to be terminated in a short period of time. In other words, resources (e.g. sockets) of the metadata server may be blocked because of a network failure.
As an example, a method for selecting a storage node and a storage device based on a query and a reply is suggested in a U.S. patent application of Amazon Technologies, Inc. (application Ser. No. 14/733,887, Publication No. US 2015/0269239) in which a data is divided to form partitions in a distributed storage environment; respective partitions are stored in distributed storage nodes, and respective partition replicas are also in the distributed storage nodes.
As another example, a method for allocating and migrating resources in a distributed environment is provided in a U.S. patent application of Alcatel Lucent USA Inc. (application Ser. No. 14/100,081, Publication No. 2015/0163157) in which a resource allocation parameter and a resource migration parameter are included in a resource request in order to allocate and migrate cloud resources in a distributed cloud system, and information for specifying a subject and position of cloud resource allocation and a location to and from which resources are migrated are provided in response to the parameters.
However, the above-described technologies do not disclose a method of effectively performing interaction between machines connected in a distributed file system via a network while having the above-described constraints.